The subject application relates to imaging and more particularly image based tracking. The subject application further relates to color-based image analysis.
Color values for pixels in an image can be described utilizing a multi-dimensional, e.g., three-dimensional coordinate system. The coordinate system used to represent the color information is commonly referred to as a color space. One common color space is Red, Green, and Blue. There are many other common three-dimensional color spaces such as YCbCr, YUV, HSV, and HSL color spaces, as well as color spaces in other dimensional orders such as RGBY and CMYK color spaces.
In general, it is possible to perform conversions, e.g., lossless or near lossless conversions, between color spaces (conversions between same-order color spaces are lossless whereas conversions between different order color spaces may have some redundancy or informational loss). Transforms between color-spaces may often be specified by a transform function/algorithm or a look-up table. Rotations and color space transformations are already widely available for various purposes and applications and can be easily implemented utilizing conventional processing technology.
While imaging analysis and other operations can be performed in any of the color spaces, the ease of doing so (e.g., mathematical complexity) can often vary greatly between different color spaces. Moreover, certain color spaces may provide better visualizations for a particular application such as tracking of an object in an image. Thus, preferred color spaces may often depend on the imaged scene and the analysis/operation that needs to be performed.
There are many other objective and subjective factors which may also influence selection of a preferred color space. For example, increasing color saturation in HSV is mathematically much easier than in YCbCr because saturation is already an independent dimension. Notably, in the forgoing example, a reduced mathematical load can improve processor performance and simplify the analysis/operation. By way of another example, in some imaging applications where visual tracking of an object is performed by an observer it may be preferable to transform the image color-space so as to enhance the ability of the observer to visualize/track the object (such as by increasing a visibility of the object, for example, relative to a background).
Describing color in a three-dimensional color system is analogous to describing a point in three-dimension space. Most people think of a three dimensional Cartesian coordinate system when thinking of a point in space. Like RGB, it is easy to comprehend. Depending on the problem one is trying to solve, it might not be the best coordinate system. For example, radial or spherical coordinate systems might be better for working with cylinders or spheres.
Raytheon's Multi-Spectral Targeting System (MTS) system utilizes a tracker module that only operates with pixel values in one dimension to facilitate efficient processing/analysis (in particular, the MTS system utilizes the luma channel of YCbCr from HDTV standards ITU-601 or ITU-709). This greatly simplifies the tracker and reduces throughput required.
The YCbCr color space is widely used in consumer electronics. One example is the interface known as “Component Video”. That interface uses three analog coaxial lines that are red, green, and blue in color. This interface was common on HD electronics before HDMI became the standard. HDMI can also transfer data in the same color space, only using digital values instead of analog for the interface.
In this color space, the Y component of the pixel (the green cable in the Component Video interface) contains all the intensity information on the pixel. The Cr and Cb contain the color information. It is possible to produce a black and white image by just using the Y portion of the signal. This is basically equivalent to driving the color saturation of the pixel to 0. It is also equivalent to setting the Cr and Cb values to midscale and leaving them hooked up.
For monochrome imaging sources, e.g., video sources, there is no loss of information because the source data only uses one of the three dimensions in the YCbCr colorspace. In the YCbCr colorspace, the Cb and Cr are equal and midscale. However, with a color pixel there can be a loss of information. The amount of loss is dependent on the exact hue of the pixel in question. For ITU-601 the luma channel is defined as:Y=0.299*Red+0.587*Green+0.114*Blue
Thus, it is possible to have a target and a background with different values in YCbCr space that have equivalent values in Y only (for example, a pure red target on a green background that is about twice the value). This is an extreme case, which is possible, but not likely. A more typical scenario is shown in FIGS. 1A and 1B below. In these images the human eye can pick out the blue color (displayed image) very easily but the gray monochrome representation of the target (tracker image) is much more difficult to visualize. The tracker module processing is impacted in a similar way by the reduced contrast of the monochrome image data.
This scenario is possible in on real imagery. It's not just with the example shown in FIGS. 1A and 1B. For example, many red targets can have problems as well. In the YCbCr colorspace, any unsaturated pixel value from black to white, including all the gray shades in between, will not exhibit this difference in perception. This is because there is no color saturation in the pixel values. Therefore the monochromatic tracker and the viewer receive the same information. Moreover, in the YCbCr colorspace there is only a small difference in perception with green values. This is mostly because the definition of the YCrCb uses a high percentage of green in the Y component. Also, in the YCbCr colorspace, target objects typically exhibit the greatest loss of contrast since target objects may include, e.g., large quantities of red and blue (including combinations such as purple). Because with those targets, information in the Cr and Cb will be at their greatest and the tracker does not receive it. To simulate this loss of contrast in real time, it should be possible to select DTV w/IC and then set the color saturation in a display to 0 (e.g., by setting the color saturation in a video encoder to 0). The video displayed should be equivalent to the video a monochromatic video tracker would receive via the Y channel.
The video tracking (AVT) subsystem in MTS is monochrome only. The Y portion of the TU output video is routed to the tracker. The entire YCrCb is output on the HD-SDI SMPTE 292. In MTS, all black and white sensors (MWIR, SWIR, DTV w/VC filter) have no Cr or Cb signal. So, in effect, the tracker “sees” exactly the same thing the sensor operator sees. The same is not true for DTV w/IC. In this configuration, the operator is provided with color information that the tracker does not receive. Because of this, some targets might appear to have sufficient contrast for AVT to work, when in reality it may be a very poor target. This is possible because the human judgment includes the color information. Sending color information to the AVT, however, isn't practical or efficient. Although getting the data to the tracker is trivial, redesigning an existing monochrome tracker to use such data would be very complicated. The tracker would have to understand the pixel value in the three dimensional color space instead of the single dimension intensity space. Moreover, increased tracker complexity would result in a higher processor load resulting in reduced efficiency. Speed is often key when it comes to tracking systems.
Thus, there exists a need for improved systems and methods for tracking an object in an imaged scene.